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Frustrated transition metal compounds in which spin-orbit coupling (SOC) and electron corre¬ 
lation work together have attracted much attention recently. In the case of 5d transition metals, 
where SOC is large, jeff =1/2 bands near the Fermi level are thought to encompass the essential 
physics of the material, potentially leading to a concrete realization of exotic magnetic phases such 
as the Kitaev spin liquid. Here we derive a spin model on a triangular lattice based on jeff =1/2 
pseudo-spins that interact via antiferromagnetic Heisenberg (J) and Kitaev (K) exchanges, and 
crucially, an anisotropic (F) exchange. Our classical analysis of the spin model reveals that, in 
addition to small regions of 120°, Z 2 / dual-Z 2 vortex crystal and nematic phases, the stripy and 
ferromagnetic phases dominate the J-K-F phase diagram. We apply our model to the bd transi¬ 
tion metal compound, Ba 3 lrTi 209 , in which the Ir^^ ions form layered two-dimensional triangular 
lattices. We compute the band structure and nearest-neighbor hopping parameters using ah-initio 
calculations. By combining our ab-initio and classical analyses, we predict that Ba 3 lrTi 2 09 has a 
stripy ordered magnetic ground state. 


I. INTRODUCTION 

Transition-metal compounds in which electrons occupy 
(i-orbitals have proven to be a vast playground for ex¬ 
otic and interesting physics. The non-trivial combina¬ 
tion of lattice geometry, crystal field strength, spin-orbit 
coupling (SOC) and Coulomb repulsion between elec¬ 
trons in transition metals can conspire to produce novel 
ground states, including a plethora of magnetic order 
Such materials, for instance the honeycomb family of 
iridate^I^®^, have drawn attentiorP^M^ owing to the pos¬ 
sibility of a concrete realization of Kitaev’s exactly solv¬ 
able spin-1/2 modeP^. 

Recently, a general model of the layered honeycomb 
iridates was derived in which jeff = 1/2 states near the 
Fermi level captured the essential physic^^. Arising from 
a combination of strong SOC and the octahedral crys¬ 
tal field environment for each Ir^+ ion, these jeff = 1/2 
states effectively behave as spin-1/2 pseudo-spins. It was 
found that a variety of magnetic ground states could be 
realized depending on the competition between Heisen¬ 
berg (J), Kitaev {K) and symmetric off-diagonal (F) 
spin exchanges at the nearest-neighbor level. The gen¬ 
eral principles applied to the honeycomb iridates can also 
be turned to triangular iridates which can harbor similar 
spin anisotropy. 

Spins on a triangular lattice are inherently frustrated 
- potentially leading to new magnetic phases. It is 
therefore important to study the interaction between 
Jeff = 1/2 pseudo-spins on a triangular lattice which 
have a microscopic origin similar to that in the hon¬ 
eycomb iridates. Classical and quantum studies of the 
Heisenberg-Kitaev ( HK) m odel have been performed on 
the triangular latticd^^^, with possible applications to 
the triangular iridate Ba3lrTi20cpIl. It has been shown 
that the HK model hosts an interesting Z2 vortex crystal 


phase which could be realized in Ba3lrTi209; moreover, 
the presence of the Kitaev exchange destabilizes the 120° 
ordered phase of the Heisenberg model on the trianglular 
lattice. We will argue, however, that this model is incom¬ 
plete and does not describe the physics of Ba3lrTi209; a 
more comprehensive approach is required. 


t? 



FIG. 1. (Color online) (a) The major nearest-neighbor hop¬ 
ping channels in the limit of ideal octahedra. The parameters 
ti and t 3 describe two distinct d-orbital overlaps, while t 2 in¬ 
cludes both d-orbital overlap and oxygen mediated hoppings. 
The green, blue and red dotted lines label the x, y and z bond 
types according to the convention a/3{'y) in Eq. 0. 

In this paper, we derive a spin model that is 
generic to layered triangular iridate compounds such as 
Ba3lrTi2 09. Our derivation is founded upon the reason¬ 
able assumption that electron interaction strength {U) 
and Hand’s coupling (Jr) provide the largest energy 
scales in the system, with SOC being the next largest 
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energy scale, and electron hopping between d-orbitals as 
the smallest energy scale. In addition, we assume that 
the symmetries of an ideal triangular lattice of octahe- 
dra are respected in our derivation. From this foundation 
we derive a nearest-neighbor spin model with three spin 
exchanges: Heisenberg (J), Kitaev {K) and a symmet¬ 
ric off-diagonal exchange (F). We map out the classical 
magnetic ground state phase diagram through a com¬ 
bination of Luttinger-TiszsP^ and classical Monte Carlo 
techniques and discuss its intricacies. 

We next apply our general model to the triangular 
iridate Ba 3 lrTi 2 09 by first performing ab-initio calcu¬ 
lations to determine its band structure with and with¬ 
out SOC. We then demonstrate that the states near the 
Fermi level are well described by jeff = 1/2 states, and 
we further estimate the nearest-neighbor tight-binding 
parameters. Consequently, we can estimate the region of 
the phase diagram that pertains to Ba 3 lrTi 209 (with and 
without the effects of octahedral distortion) and predict 
that it should exhibit stripy magnetic order in its ground 
state. Finally, experimental methods to test our predic¬ 
tion are also discussed. 

Our paper is organized in the following manner. We 
derive our general jeff = 1/2 spin model on a triangular 
lattice and analyze its classical phase diagram in Sec¬ 
tion IL In Section III we discuss the crystal structure 
of Ba 3 lrTi 209 , present the results of our ab-initio cal¬ 
culations, estimate the strengths of the spin exchanges 
from ab-initio tight-binding parameters, and predict the 
ground state for Ba 3 lrTi 209 using our model. In Sec¬ 
tion IV we examine the effects of octahedral distortion 
in Ba 3 lrTi 209 within our spin model. Finally, in Section 
V, we discuss and summarize the ramifications of these 
results and suggest possible experimental tests. 


II. DERIVATION OF jeff=l/2 SPIN MODEL ON 
A TRIANGULAR LATTICE & 
LUTTINGER-TISZA AND CLASSICAL MONTE 
CARLO ANALYSIS 


We begin by constructing a model of a triangular lat¬ 
tice of isolated Ir^+ ions which are surrounded by ideal 
oxygen octahedra as in Fig. The local octahedral 

crystal field will split the Ir^+ bd levels into t 2 g and Cg 
states. The large SOC of the Ir^+ ions will further split 
the t 2 g manifold into completely filled jeff = 3/2 states 
and half-filled jeff = 1/2 states. Since the strong on¬ 
site Coulomb interactions localize these jeff = 1/2 states, 
we can then consider the effective physics to be cap¬ 
tured by interacting jeff = 1/2 pseudo-spins. To arrive 
at this picture concretely, we start from a microscopic 
Hamiltonian which includes nearest-neighbor d-orbital 
hoppings as perturbations on top of an on-site Kanamori 
HamiltoniaiP^ (see the Supplementary Material for more 
details). There are two types of d-orbital hopping at the 
nearest-neighbor level as shown in Fig. direct overlap 
between d-orbitals on Ir^+ sites (denoted by ti and ^ 3 ), 


and oxygen mediated hopping via p-orbitals (denoted by 

ti). 

The nearest-neighbor spin model takes the form 


H = 


E 


JSi ■ Sj + KSJS] + r(5“5f + sfsf) 


( 1 ) 

where i, j denote Ir^+ sites, and is a jeff = 1/2 spin 
operator at a site with components 5'f {a = x, y, z). The 
exchanges J and K are Heisenberg and Kitaev exchanges 
respectively, while F is a symmetric off-diagonal exchange 
(exact expressions can be found in the Appendix). We 
identify each bond as one of either yz{x)^ zx{y) or xy{z) 
bonds, labeled by aP{'y). The spin components that in¬ 
teract on each bond depend on the bond type aP{'y). 

We studied the phase space of the spin model using a 
combination of Luttinger-Tisza (LT) and classical Monte 
Carlo (CMC) analyses in order to determine the ground 
state for varying exchange parameters. We employed 
the simulated annealing technique in our CMC calcula¬ 
tions, whereby the transition probability from state to 
state is determined by a temperature parameter that 
is slowly lowered until there is no improvement in the 
ground state energy. The classical approaches are con¬ 
sidered simultaneously in Fig. The LT and CMC 
analyses were found to agree with each other in both 
energy and spin configuration, except for small regions 
of the phase diagram computed by CMC (bounded by 
the dotted line toward the boundary), where LT could 
not identify the exact classical ground state. The ex¬ 
changes are parameterized in spherical coordinates such 
that J = sin 0 cos (j)^ K = sin 0 sin (j) and F = cos 0 where 
0 G [ 0 , 7 r], and 0 G [0,27r). The Heisenberg-Kitaev (HK) 
model is recovered in the limit in which 0 = 7r/2, where 
F = 0. We find six different magnetic orderings: fer- 
romagnet (FM), stripy (ST), 120° order (120), nematic 
(N), Z 2 vortex crystal and dual-Z 2 vortex crystal. We 
depict the FM, ST, and 120 orders in the insets of Fig. 

while nematic, Z 2 vortex crystal and dual-Z 2 vortex 
crystal phases are depicted in the Supplementary Mate¬ 
rial. A thorough description of Z 2 and dual-Z 2 orders 
can be found in Refs. l^andlSOl 

The boundaries of both phase diagrams in Fig. co¬ 
incide, and show the classical ground states of the HK 
model (F = 0 in Eq. 0 ). We first highlight some special 
points. The Heisenberg model (J = 1) hosts 120° order, 
agreeing with the known ground state on a triangular lat¬ 
tice. We also find that perturbing away from the Heisen¬ 
berg model by adding K tends to destabilize 120° order 
toward the previously discussed Z 2 vortex crystal phase. 
The K = 1 point represents the Kitaev model on the 
triangular lattice. Here we find that the spins tend to or¬ 
der antiferromagnetically in chains along one of the three 
principal directions of the triangular lattice, with each 
chain decoupled from the rest. By flipping all the spins 
in any of the chains, the energy remains the same and so 
this state is highly degenerate - exponential in the size of 
the system. This classical ground state degeneracy has 
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FIG. 2. (Color online) Combined Luttinger-Tisza (LT) and Classical Monte Carlo (CMC) phase diagrams of the triangular 
lattice J-K-T model for F > 0 (a) and F < 0 (b). The angles 0 and 0 denote the radial and azimuthal angles respectively. 
There are six phases represented: ferromagnet (FM), stripy (ST), 120 order (120), nematic (N), Z 2 vortex (Z 2 ) and dual-Z 2 
(dZ 2 ). Real space spin representations of FM, ST and 120 orders are shown in the insets, while nematic, Z 2 vortex crystal and 
dual-Z 2 vortex crystal phases are shown in the Supplementary Material. Phases bounded toward the edge (F = 0) by a dotted 
line are computed using CMC. The region marked by the white star is the relevant parameter regime for Ba 3 lrTi 209 predicted 
by our combined ab-initio and classical analysis. 


recently been analyzed and has been shown to be lifted 
by quantum fluctuations through an order-by-disorder 
mechanism, introducing a coupling between spms of next- 
nearest neighboring antiferromagnetic chains^. Overall, 
the boundary of the phase diagram is in agreement with 
the previous Monte Carlo studies of the HK model on 
the triangular lattice. 

However, when adding F, whether positive or negative, 
to the Heisenberg model, we find that 120° order emerges. 
For positive F, we find that 120° order persists until the 
degenerate F = 1 point, where 120°, FM and ST orders 
meet. For the Kitaev model, on the other hand, the 
introduction of small |F| immediately gives ST order as 
the magnetic ground state. The majority of the phase 
diagram upon the inclusion of F is dominated by stripy 
or ferromagnet phases. 


III. APPLICATION TO Ba3lrTi209 

Ba 3 lrTi 209 has a layered triangular lattice structure 
composed of Ir'^^ ions with a electron configuration 
surrounded by oxygen octahedra. It belongs to space- 
group P6smc, possessing a mirror plane and a screw 
axis along the c-direction, connecting the top and bottom 
halves of the unit cell. The bottom half of the unit cell 
is shown bounded by the rectangular box in Fig[^. The 
Ir 02 layers are separated by two Ti 02 layers, with the 
Ir^+ ions face-sharing their octahedra with the Ti^+ oc¬ 
tahedra directly above them along the c-axis. The Ba^+ 
ions populate the unit cell among the Ir 02 and Ti 02 lay¬ 


ers. There are two Ir^+ ions belonging to the primitive 
unit cell, each of which belongs to different Ir 02 layers 
that form a triangular lattice, shown in Fig [^. The 
distortion of the local octahedra, depicted in Fig [^, is 
discussed in the next section. 

The local environment for the electrons at each site, 
with an octahedral crystal field, and large SOC, is remi¬ 
niscent of that in Na 2 lr 03 , Li 2 lr 03 and (a-RuCl 3 . There¬ 
fore, we may expect that Ba 3 lrTi 209 can be described 
by similar physical principles involving pseudo-spin jeff = 
1/2 states near the Fermi level. Unlike in honeycomb iri- 
dates, however, Ba 3 lrTi 209 forms triangular lattice lay¬ 
ers and the oxygen octahedra surrounding the Ir^+ ions 
are not edge-shared. Here we explain that the jeff =1/2 
spin model derived in the previous section captures the 
essential physics of Ba 3 lrTi 209 . To this end, we perform 
a series of band structure calculations for Ba 3 lrTi 209 
and determine the nature of the electronic states near 
the Fermi level. 

We used OpenM}(P^, which implements the linear- 
combination-of-pseudo-atomic-orbitals method, to com¬ 
pute the band structure of Ba 3 lrTi 209 with and without 
SOC. A non-collinear DFT scheme and a fully relativis¬ 
tic j-dependent pseudopotential are used to treat SOC, 
with the Perdew-Zunger parameterization of the local 
density approximation (LDA) chosen for the exchange- 
correlation functionaP^. We chose an energy cutoff of 
300 Ry for the real-space sampling and used an 8 x 8 x 3 
k-giid for the Brillouin zone sampling. We used the 
maximally-localized Wannier orbitals method^ imple¬ 
mented in OpenM}(P^ to obtain the tight-binding Hamil- 
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FIG. 3. (Color online) (a) Half of the Ba 3 lrTi 209 primitive 
unit cell (bounded by the rectangular box) translated once in 
the a-direction featuring Ba (pale-green), Ti (blue), O (red) 
and Ir (yellow) atoms. There are two Ir^^ ions in the unit cell, 
each belonging to different layered triangular lattices as in 
(b), with each site surrounded by oxygen octahedra. Nearest- 
neighbor Ir atoms belonging to different unit cells are con¬ 
nected by the black double-arrowed line. The Ir^^ octahedra 
are face-shared with Ti^^ octahedra. (c) Representation of 
distorted octahedra featuring two new hopping parameters, 
t 2 and t 2 ? leading to a Dzyaloshinksii-Moriya (DM) exchange 
interaction in Eq. §. 

tonian for Ir t 2 g orbitals. 

The results of our calculation are shown in Fig[^ The 
calculated band structure without SOC is shown in Fig 

alongside the projected density of states (PDOS). The 
bands near the Fermi level are composed of p-orbitals 
from oxygen and t 2 g orbitals coming from the Ir^+ ions. 
A gap of approximately 1 eV separates these bands from 
the empty Ti d-bands. We note that the bandwidth is 
relatively small, reflecting the large distance between Ir^+ 
sites. 

To understand the effects of SOC, we also computed 
the band structure including SOC, shown in Figwith 
the bands and PDOS projected onto the jeff = 1/2 and 
3/2 states. We find that the t 2 g bands are split by SOC 
into well-separated jeff =1/2 (pink) and jeff = 3/2 (teal) 


states. The five valence electrons will serve to completely 
fill the jeff = 3/2 states and half-fill the jeff = 1/2 
states. We can therefore expect the effective physics 
of Ba 3 lrTi 209 to be well described by the pseudo-spin 
Jeff = 1/2 model derived in the previous section. 

Having established the jeff = 1/2 nature of 

Ba 3 lrTi 2 09 , we can further estimate the hopping param¬ 
eters coming from the major hopping channels depicted 
schematically in Fig [^. We again use the maxially- 
localized Wannier orbital method to estimate the nearest- 
neighbor hopping parameters. In the limit of ideal lo¬ 
cal octahedra, we find that the nearest-neighbor hopping 
parameters are ti = 7.4 meV, t 2 = —23.5 meV and 
ts = —119 meV. All further neighbor hopping param¬ 
eters are found to be small (< 5 meV), therefore, a tight 
binding model including only nearest-neighbor parame¬ 
ters will aptly describe the kinetic microscopies. 

From our ab-initio calculations, we can estimate the 
strength of the exchanges knowing the strengths of the 
nearest-neighbor tight binding parameters. Taking on¬ 
site Coulomb repulsion to he U = 2.0 eV and fixing 
the ratio of Hand’s coupling to U as Jh/U = 0.2, we 
find J 2 meV, iF 4 meV and F —2 meV 
for Ba 3 lrTi 209 . Normalizing the exchange couplings 
by = x/J2 + K2TT2 (so that + F^ = 1), 

we equivalently find J/N 0.36, K/N 0.86 and 
F /N = —0.36, with the Kitaev exchange being the dom¬ 
inant exchange at the nearest-neighbor level. We also 
point out that F is non-zero and negative. This area of 
phase space is marked by the white star in Fig. and 
falls into the stripy region of magnetic order. 

Our analysis has been performed under the most gen¬ 
eral considerations imposed by the symmetries of the lat¬ 
tice and the local electron environment at each site, ar¬ 
riving at the model in Eq. 0- When F = 0, our model 
reduces to the HK model; however, the HK model is in¬ 
sufficient to describe Ba 3 lrTi 209 as seen in the significant 
F that is of the same strength as the Heisenberg exchange 
J. Indeed, even a relatively small F can act to unsettle 
the phases on the boundary of the phase diagram. More¬ 
over, all layered triangular systems of this kind can be 
treated with our model and, barring fine-tuning of the 
tight binding parameters, F is not zero in general. 


IV. EFFECTS OF OCTAHEDRAL DISTORTION 

We also investigated the presence of the distortion in 
the oxygen octahedra, which breaks inversion symmetry 
about the Ir-Ir bond center. Such a distortion will induce 
two inequivalent oxygen mediated hoppings, t 2 and 
shown in Fig. If, as well as other potential hoppings. 
From our ab-initio calculations, we estimate that t 2 = 
— 13 meV and ^2 = —32 meV. In the case of Ba 3 lrTi 209 , 
we only consider these hopping parameters since our ab- 
initio calculations suggest that other hopping channels 
created by the distortion have small amplitudes (less than 
half of ^ 2 ). 
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FIG. 4. (Color online) Band structure calculations of Ba 3 lrTi 209 performed using OpenMX. (a) Band structure without SOC 
and projected density of states including Ir t 2 ^-, Ti d-, and O p-orbitals. The bands near the Fermi level are composed of 
contributions from Ir^^ t 2 g orbitals and O p-orbitals. (b) Band structure with SOC and jeff projected density of states. The 
bands are projected onto jeff =1/2 and jeff = 3/2 states using maximally-localized Wannier orbitals. The relative contribution 
of each state is represented by the intensity of the color. The t 2 g bands are well separated into jeff = 1/2 (pink) and jeff = 3/2 
(teal) bands. 


The main result of the non-ideal local crystal envi¬ 
ronment is to introduce a Dzyaloshinskii-Moriya (DM) 
exchange term to the spin model. It also breaks apart 
the Heisenberg-Kitaev exchanges into three anisotropic 
exchanges: 


-f^distorted — ^ ^ 

a/3(7)e(ii) 


+ J^S'f S'f + J'>'5757 

J J t' J 


+ r(5f 5“) 


( 2 ) 


where the bond labeling convention remains the same 
and D denotes the DM exchange (see the Supplemen¬ 
tary Material for exchange expressions). Estimating the 
strengths of these new exchanges, we find 2 meV, 

jy 2 meV, 6 meV, F ^ —2 meV and D 2 meV. 
When we return to the ideal octahedra limit, ^ J-\-K 
and J^,jy ^ J. 

Using the above parameters to model the distortion ef¬ 
fects, the magnetic ground state is found to be stripy. We 
next analyzed the effect of increasing \D\ and |F| while 
keeping J^, and fixed. Combining LT and CMC 
analyses, we find that the DM exchange tends to stabilize 
stripy order. Indeed, upon varying |D|/J^ and |r|/J^ in¬ 
dependently in the interval [—100,100] x [—100,100], we 
find only stripy order. Therefore, we predict that the 
ground state of Ba 3 lrTi 2 09 harbors a stripy magnetic 
ordering pattern. 


V. DISCUSSION AND CONCLUSION 

In summary, we have derived a general jeff =1/2 spin 
model on a triangular lattice based on lattice symmetry 


considerations and the interplay between strong electron 
correlations, large SOC and a local octahedral crystal 
field environment. By employing Luttinger-Tisza and 
classical Monte Carlo, we have identified six distinct mag¬ 
netic ground states supported by our model. We then de¬ 
termined the electronic band structure and nature of the 
states near the Fermi level in the layered triangular com¬ 
pound Ba 3 lrTi 209 from ah-initio calculations, and found 
that it is well described by the pseudo-spin jeff =1/2 pic¬ 
ture. Applying our ah-initio calculations to our general 
spin model, we predict that Ba 3 lrTi 209 hosts a stripy 
magnetic ground state near the Kitaev limit. Our pre¬ 
diction for stripy order as the magnetic ground state of 
Ba 3 lrTi 2 09 can be verified by neutron scattering. A 
magnetic peak at ordering wave vector M in the Bril- 
louin zone would correspond to stripy order according to 
our analysis. 

A previous experimental study of Ba 3 lrTi 2 0cpll mea¬ 
sured zero-field-cooled and field-cooled magnetic suscep¬ 
tibility in the temperature range of 2-400 K, as well 
as heat capacity measurements from 0.35-295 K. No 
long range magnetic ordering was reported in this ex¬ 
perimental study; however. X-ray powder diffraction of 
Ba 3 lrTi 2 09 at room temperature revealed large site shar¬ 
ing between Ir^+ and TU+ ions of (37 ± 10)%. It is pos¬ 
sible that the site sharing between Ir^+ and TU+ ions 
could lead to a nonmagnetic dilution of the triangular 
layers. Studies of nonmagnetic dilution have been re¬ 
cently u nderta ken on the honeycomb iridates AIr 03 (A 
= Li, Na)P^Eo|^ with the observation of possible spin-glass 
behavior. Further studies of nonmagnetic dilution on tri¬ 
angular lattices, which may enhance frustration and in¬ 
duce a correlated paramagnetic phase, can be important 
both in understanding material properties and in real- 
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izing novel phases. Based on our work, we expect that 
stripy magnetic ordering will be observed in a pure sam¬ 
ple. 


The resulting perturbed Hamiltonian is projected onto 
the Jeff = 1/2 subspace since the SOC is large. In terms 
of the Jeff = 1/2 spin operators, we arrive at the effective 
Hamiltonian 
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i^ideal = E > 

a/3i-f)e{ij) 

(A3) 

written in the main text, with 


y / + 2 ^ 3 ) 2{ti - {2ti + 

27 V U-3Jh ^ U-Jh ^ U + 2Jh ) 

(A4) 


8Jg / {ti - tzY - 3tl \ 
9 \{U-3Jh){U-Jh)) 


(A5) 


16Jff / ^ 2(^1 — ts) \ 

9 [(U-^JhKU-Jh))' 


(A 6 ) 


Appendix A: Derivation of jeff =1/2 spin model 
without octahedra distortion 

We derive the spin model in the limit of ideal octahedra 
by performing a strong coupling expansion in the limit 
where U^Jh ^ ^so ^ ^5 with U being the strength of 
the Coulomb interaction, Jh is Hand’s coupling, \so is 
the SOC strength and t are the tight binding parameters. 
We start by considering an on-site Kanamori Hamilto¬ 
nian, 

Ho = Y1 - 5)2 - 2JHSf - , 

(Al) 

where i denotes the Ir^+ site index, and Si and Li are 
the total number, spin and angular momentum operator 
on the site. 

To derive an effective spin Hamiltonian, we con¬ 
sider electron hopping through major nearest-neighbor 
d-orbital hopping channels as small perturbations on top 
of Hq within the framework of second order perturba¬ 
tion theory. The kinetic part of the Hamiltonian (along 
a z-bond for instance) takes the general form 

Tij + = E {dlAdjp, + , 

a,a,l3 

where creates a d-orbital electron (a, (3 = yz^ zx^ xy) 
on site i with spin a, and is the hopping matrix along 
the 2 )-bond connecting the orbitals on the two nearest- 
neighbor sites. In the ideal octahedra limit, the hopping 
matrix is restricted by C 2 rotation symmetry along the 
bond connecting the two sites and C 2 Z symmetry about 
the bond center to the form (in the \yz)^ \zx)^ \xy) basis): 

/ti t2 0\ 

T^=\t2 ti 0 . (A 2 ) 

VO 0 u) 


Appendix B: Expression for spin exchanges when 
including octahedra distortion effects 

As discussed in the main text, the distorted octahedra 
add two new hopping parameters t 2 and In terms of 
the hopping matrix of the previous section, we have (in 
the \yz)^ jzx), \xy) basis): 

Al ^2 o\ 

^distorted = 02*1 0 . (Bl) 

VO 0 hj 

This non-symmetric hopping matrix introduces a 
Dzyaloshinksii-Moriya exchange in the effective Hamilto¬ 
nian and also splits the Heisenberg and Kitaev exchanges 
found in the ideal limit. Along a z-bond (sites I and 2), 
the Hamiltonian takes the form, 

^distorted = + JyS\Sl + J,SISI+ (B2) 

D ( 5 f - sis^) + r + SIS ^), 

with the new exchange parameters given by, 

4 / 2t^ — 4tit3 — 2^2 T ^ 2^2 T t^2 T ^^3 

3 (2t^ + 4tit3 + ^ 2^2 — t' 2 ^ 

U-3Jh 

( —2ti + ^2 — ^2 — t3)(2ti + ^2 — ^2 + ts) \ 

U + 2Jh a 

4 / 2t^ — T ^2 T ^ 2^2 — 2^2^ T 2^3 

3 (2t^ + — t2 *2*2) 

U-3Jh 

{ — 2ti + ^2 — *2 — *3)(2tl + ^2 — *2 + *3) \ 

U + 2Jh /’ 
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4 / + 2tit3 + 2^2 + ^^2^2 + 2^2^ — ^3 

^ ^ ^v u -Jh 

3 (—3ti — 2tits + 3t2t2 — tl) 
U-3Jh 

+ 4tii3 + il — 4" 4" ^3 

f/ + 2Jff 7’ 

r=^7 (*2+ i2)(ii-^ 3 ) 

9 {U -^Jh){U - JhY 


D= -^{t2-t'^) 


3(ti + ts) — ts 2ti + ts \ 

U-3Jh ^ U-Jh^ U^2Jh)' 


For Ba 3 lrTi 209 , we find the following tight-binding 
parameters from ab-initio calculations: ti = 7.4 meV, 
t 2 = —13 meV, ^2 = —32 meV and ts = —119 meV. 
Taking on-site Coulomb repulsion to he U = 2.0 eV and 
fixing JhIU = 0.2, we find ^ 2 meV, 2 meV, 

jz Q meV, r — 2 meV and D 2 meV. We return 
to the ideal octahedra limit when ^2 = ^2 = — 23.5 meV. 
In this limit we find J 2 meV, K A meV and T ^ — 2 
meV. 


Appendix C: Real Space Spin Representation of 
Phases 


Real space spin representations of nematic, Z 2 and 
dual-Z 2 phases from CMC are shown in Fig. These 
phases are found near the boundary of the phase dia¬ 
grams in Fig. where |r| is small. The nematic phase. 
Fig. 1^, is found near the Kitaev limit, and character¬ 
ized by uncorrelated antiferromagnetic chains that point 
along one of the three principal directions of the trian¬ 
gular lattice. The Z 2 vortex crystal and dual-Z 2 vortex 
crystal phases are shown in Figures It and respec¬ 
tively, and are discussed in Refs. [29]and|30l The Z 2 vor¬ 
tex crystal phase is incommensurate and features large 
vortices that span the lattice. It emerges upon pertur¬ 
bation of the Heisenberg model on the triangular lattice 
with a small Kitaev term. The dual-Z 2 vortex crystal 
phase is related to the Z 2 vortex crystal phase via Klein 
duality - a four-sublattice transformation of the spins 
that maps J —J and K ^ 2J -\- K. 
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FIG. 5. (Color online) Real space spin configuration examples of nematic (a), Z 2 vortex crystal (b) and dual-Z 2 vortex crystal 
phases. The orientation of each spin into (out of) the triangular plane is represented by the blueness (redness) of the color of 
the spins. The colored boundary around each phase corresponds to the color used to label the phase in Fig. [^of the main text. 
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